Component attention network for multimodal dance improvisation recognition

Dance improvisation is an active research topic in the arts. Motion analysis of improvised dance can be challenging due to its unique dynamics. Data-driven dance motion analysis, including recognition and generation, is often limited to skeletal data. However, data of other modalities, such as audio, can be recorded and benefit downstream tasks. This paper explores the application and performance of multimodal fusion methods for human motion recognition in the context of dance improvisation. We propose an attention-based model, component attention network (CANet), for multimodal fusion on three levels: 1) feature fusion with CANet, 2) model fusion with CANet and graph convolutional network (GCN), and 3) late fusion with a voting strategy. We conduct thorough experiments to analyze the impact of each modality in different fusion methods and distinguish critical temporal or component features. We show that our proposed model outperforms the two baseline methods, demonstrating its potential for analyzing improvisation in dance.Comment: Accepted to 25th ACM International Conference on Multimodal Interaction (ICMI 2023

Explainable Predictive Maintenance

Explainable Artificial Intelligence (XAI) fills the role of a critical interface fostering interactions between sophisticated intelligent systems and diverse individuals, including data scientists, domain experts, end-users, and more. It aids in deciphering the intricate internal mechanisms of ``black box'' Machine Learning (ML), rendering the reasons behind their decisions more understandable. However, current research in XAI primarily focuses on two aspects; ways to facilitate user trust, or to debug and refine the ML model. The majority of it falls short of recognising the diverse types of explanations needed in broader contexts, as different users and varied application areas necessitate solutions tailored to their specific needs. One such domain is Predictive Maintenance (PdM), an exploding area of research under the Industry 4.0 \& 5.0 umbrella. This position paper highlights the gap between existing XAI methodologies and the specific requirements for explanations within industrial applications, particularly the Predictive Maintenance field. Despite explainability's crucial role, this subject remains a relatively under-explored area, making this paper a pioneering attempt to bring relevant challenges to the research community's attention. We provide an overview of predictive maintenance tasks and accentuate the need and varying purposes for corresponding explanations. We then list and describe XAI techniques commonly employed in the literature, discussing their suitability for PdM tasks. Finally, to make the ideas and claims more concrete, we demonstrate XAI applied in four specific industrial use cases: commercial vehicles, metro trains, steel plants, and wind farms, spotlighting areas requiring further research.Comment: 51 pages, 9 figure

Detecting Changes of a Distant Gas Source with an Array of MOX Gas Sensors

We address the problem of detecting changes in the activity of a distant gas source from the response of an array of metal oxide (MOX) gas sensors deployed in an open sampling system. The main challenge is the turbulent nature of gas dispersion and the response dynamics of the sensors. We propose a change point detection approach and evaluate it on individual gas sensors in an experimental setup where a gas source changes in intensity, compound, or mixture ratio. We also introduce an efficient sensor selection algorithm and evaluate the change point detection approach with the selected sensor array subsets

Change detection in metal oxide gas sensor signals for open sampling systems

This thesis addresses the problem of detecting changes in the activity of a distant gas source from the response of an array of metal oxide (MOX) gas sensors deployed in an Open Sampling System (OSS). Changes can occur due to gas source activity such as a sudden alteration in concentration or due to exposure to a different compound. Applications such as gas-leak detection in mines or large-scale pollution monitoring can benefit from reliable change detection algorithms, especially where it is impractical to continuously store or transfer sensor readings, or where reliable calibration is difficult to achieve. Here, it is desirable to detect a change point indicating a significant event, e.g. presence of gas or a sudden change in concentration. The main challenges are turbulent dispersion of gas and the slow response and recovery times of MOX sensors. Due to these challenges, the gas sensor response exhibits fluctuations that interfere with the changes of interest. The contributions of this thesis are centred on developing change detection methods using MOX sensor responses. First, we apply the Generalized Likelihood Ratio algorithm (GLR), a commonly used method that does not make any a priori assumption about change events. Next, we propose TREFEX, a novel change point detection algorithm, which models the response of MOX sensors as a piecewise exponential signal and considers the junctions between consecutive exponentials as change points. We also propose the rTREFEX algorithm as an extension of TREFEX. The core idea behind rTREFEX is an attempt to improve the fitted exponentials of TREFEX by minimizing the number of exponentials even further. GLR, TREFEX and rTREFEX are evaluated for various MOX sensors and gas emission profiles. A sensor selection algorithm is then introduced and the change detection algorithms are evaluated with the selected sensor subsets. A comparison between the three proposed algorithms shows clearly superior performance of rTREFEX both in detection performance and in estimating the change time. Further, rTREFEX is evaluated in real-world experiments where data is gathered by a mobile robot. Finally, a gas dispersion simulation was developed which integrates OpenFOAM flow simulation and a filament-based gas propagation model to simulate gas dispersion for compressible flows with a realistic turbulence model

Analysis of Statistical Data Heterogeneity in Federated Fault Identification

Federated Learning (FL) is a setting where different clients collaboratively train a Machine Learning model in a privacy-preserving manner, i.e., without the requirement to share data. Given the importance of security and privacy in real-world applications, FL is gaining popularity in many areas, including predictive maintenance. For example, it allows independent companies to construct a model collaboratively. However, since different companies operate in different environments, their working conditions may differ, resulting in heterogeneity among their data distributions. This paper considers the fault identification problem and simulates different scenarios of data heterogeneity. Such a setting remains challenging for popular FL algorithms, and thus we demonstrate the considerations to be taken into account when designing federated predictive maintenance solutions.

Multi-Task Representation Learning

The majority of existing machine learning algorithms assume that training examples are already represented with sufficiently good features, in practice ones that are designed manually. This traditional way of preprocessing the data is not only tedious and time consuming, but also not sufficient to capture all the different aspects of the available information. With big data phenomenon, this issue is only going to grow, as the data is rarely collected and analyzed with a specific purpose in mind, and more often re-used for solving different problems. Moreover, the expert knowledge about the problem which allows them to come up with good representations does not necessarily generalize to other tasks. Therefore, much focus has been put on designing methods that can automatically learn features or representations of the data instead of learning from handcrafted features. However, a lot of this work used ad hoc methods and the theoretical understanding in this area is lacking

Stacked Ensemble of Recurrent Neural Networks for Predicting Turbocharger Remaining Useful Life

Predictive Maintenance (PM) is a proactive maintenance strategy that tries to minimize a system’s downtime by predicting failures before they happen. It uses data from sensors to measure the component’s state of health and make forecasts about its future degradation. However, existing PM methods typically focus on individual measurements. While it is natural to assume that a history of measurements carries more information than a single one. This paper aims at incorporating such information into PM models. In practice, especially in the automotive domain, diagnostic models have low performance, due to a large amount of noise in the data and limited sensing capability. To address this issue, this paper proposes to use a specific type of ensemble learning known as Stacked Ensemble. The idea is to aggregate predictions of multiple models—consisting of Long Short-Term Memory (LSTM) and Convolutional-LSTM—via a meta model, in order to boost performance. Stacked Ensemble model performs well when its base models are as diverse as possible. To this end, each such model is trained using a specific combination of the following three aspects: feature subsets, past dependency horizon, and model architectures. Experimental results demonstrate benefits of the proposed approach on a case study of heavy-duty truck turbochargers. © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). HEALTH-VINNOV